The use of packet based communication networks to transport digitized streams of audio and video is becoming increasingly common. The characteristics of these networks are such that the total delay experienced by each data packet is a function of variable delays due to physical media access, relay queuing, and choice of routes in addition to fixed propagation delays. The result is that the time difference between transmitting any two packets at the source is unlikely to be the same as that observed between their arrival at the destination. The delay variations (known as jitter) are a particular problem for a stream of multimedia packets, because the jitter can have an impact on the audiovisual quality as perceived by a human user. Delay is also a problem when using multiple related streams, because it introduces a temporal difference which is undesirable in situations where, for example, lipsynch is required. To solve the synchronization problems, one must introduce additional delay by buffering packets at or near the point of presentation. In the single stream case, this is used to smooth out jitter prior to passing data samples to an output device. For multiple related streams it is used to compensate for interstream delay differences. For real time applications such as conferencing and telephony, large delays are detrimental to interactivity. For example, in packetized voice applications, the maximum acceptable end-to-end delay is in the 150 to 250 millisecond range.
Two approaches to synchronizing data packets are known as packet preserving and time preserving. In the former, the receiver waits until the packet arrives to play the packet back, thus all packets are played. In the latter time preserving method, if a packet does not arrive within a certain time boundary, the packet is assumed to be lost and subsequent packets are played back. If the lost packet does arrive later it is thrown away. With the packet preserving approach buffer requirements are large and the end-to-end delay between transmission and playback can be high. The time preserving approach can result in a large number of uncontrolled losses. As mentioned, some applications are sensitive to end-to-end delay and most are affected by losses. Neither approach is suitable for use with a wide range of applications.
As a result there is a need for a network data stream synchronization system able to satisfy a wide range of application and user requirements and configurable to operate according to user or application specified delay versus loss tradeoffs.